{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "学习目标\n",
    "- 应用cut、qcut实现数据的区间分组\n",
    "- 应用get_dummies实现数据的one-hot编码\n"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "24b27bd4e8c0af03"
  },
  {
   "cell_type": "markdown",
   "source": [
    "# 1 为什么要离散化\n",
    "连续属性离散化的目的是为了简化数据结构，数据离散化技术可以用来减少给定连续属性值的个数。离散化方法经常作为数据挖掘的工具。"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "2b4af5c1f37ec6d8"
  },
  {
   "cell_type": "markdown",
   "source": [
    "# 2 什么是数据的离散化\n",
    "连续属性的离散化就是在连续属性的值域上，将值域划分为若干个离散的区间，最后用不同的符号或整数 值代表落在每个子区间中的属性值。\n",
    "离散化有很多种方法，这使用一种最简单的方式去操作\n",
    "- 原始人的身高数据：165，174，160，180，159，163，192，184\n",
    "- 假设按照身高分几个区间段：150~165, 165~180,180~195\n",
    "这样我们将数据分到了三个区间段，我可以对应的标记为矮、中、高三个类别，最终要处理成一个\"哑变量\"矩阵\n",
    "# 3 股票的涨跌幅离散化\n",
    "我们对股票每日的\"p_change\"进行离散化\n",
    "## 3.1 读取股票的数据\n",
    "先读取股票的数据，筛选出p_change数据"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "a5518aade11fd351"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "data": {
      "text/plain": "2018-02-27    2.68\n2018-02-26    3.02\n2018-02-23    2.42\n2018-02-22    1.64\n2018-02-14    2.05\n              ... \n2015-03-06    8.51\n2015-03-05    2.02\n2015-03-04    1.57\n2015-03-03    1.44\n2015-03-02    2.62\nName: p_change, Length: 643, dtype: float64"
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "data = pd.read_csv(\"stock_day.csv\")\n",
    "p_change= data['p_change']\n",
    "p_change"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-02-22T09:37:24.428836300Z",
     "start_time": "2024-02-22T09:37:24.421377600Z"
    }
   },
   "id": "2571ecb7eaf664ee",
   "execution_count": 11
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 3.2 将股票涨跌幅数据进行分组\n",
    "使用的工具：\n",
    "- pd.qcut(data, q)：\n",
    "    - 对数据进行分组将数据分组，一般会与value_counts搭配使用，统计每组的个数\n",
    "- series.value_counts()：统计分组次数"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "87e96bee6642ceeb"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "data": {
      "text/plain": "p_change\n(-10.030999999999999, -4.836]    65\n(-0.462, 0.26]                   65\n(0.26, 0.94]                     65\n(5.27, 10.03]                    65\n(-4.836, -2.444]                 64\n(-2.444, -1.352]                 64\n(-1.352, -0.462]                 64\n(1.738, 2.938]                   64\n(2.938, 5.27]                    64\n(0.94, 1.738]                    63\nName: count, dtype: int64"
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 自行分组\n",
    "qcut = pd.qcut(p_change, 10)\n",
    "# 计算分到每个组数据个数\n",
    "qcut.value_counts()"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-02-22T09:38:00.453937800Z",
     "start_time": "2024-02-22T09:38:00.446023400Z"
    }
   },
   "id": "971909d75ba5851d",
   "execution_count": 12
  },
  {
   "cell_type": "markdown",
   "source": [
    "自定义区间分组：\n",
    "- pd.cut(data, bins)"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "d7f8172a14a8ed98"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "data": {
      "text/plain": "2018-02-27      (0, 3]\n2018-02-26      (3, 5]\n2018-02-23      (0, 3]\n2018-02-22      (0, 3]\n2018-02-14      (0, 3]\n                ...   \n2015-03-06    (7, 100]\n2015-03-05      (0, 3]\n2015-03-04      (0, 3]\n2015-03-03      (0, 3]\n2015-03-02      (0, 3]\nName: p_change, Length: 643, dtype: category\nCategories (8, interval[int64, right]): [(-100, -7] < (-7, -5] < (-5, -3] < (-3, 0] < (0, 3] < (3, 5] < (5, 7] < (7, 100]]"
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 自己指定分组区间\n",
    "bins = [-100, -7, -5, -3, 0, 3, 5, 7, 100]\n",
    "p_counts = pd.cut(p_change, bins)\n",
    "p_counts"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-02-22T09:38:55.987832100Z",
     "start_time": "2024-02-22T09:38:55.983095500Z"
    }
   },
   "id": "8744c981e09508f9",
   "execution_count": 14
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 3.3 分组数据变成one-hot编码\n",
    "- 什么是one-hot编码\n",
    "把每个类别生成一个布尔列，这些列中只有一列可以为这个样本取值为1.其又被称为独热编码。\n",
    "- pandas.get_dummies(data, prefix=None)\n",
    "    - data:array-like, Series, or DataFrame\n",
    "    - prefix:分组名字"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "f63c92e7654f837f"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "data": {
      "text/plain": "            rise_(-100, -7]  rise_(-7, -5]  rise_(-5, -3]  rise_(-3, 0]  \\\n2018-02-27            False          False          False         False   \n2018-02-26            False          False          False         False   \n2018-02-23            False          False          False         False   \n2018-02-22            False          False          False         False   \n2018-02-14            False          False          False         False   \n...                     ...            ...            ...           ...   \n2015-03-06            False          False          False         False   \n2015-03-05            False          False          False         False   \n2015-03-04            False          False          False         False   \n2015-03-03            False          False          False         False   \n2015-03-02            False          False          False         False   \n\n            rise_(0, 3]  rise_(3, 5]  rise_(5, 7]  rise_(7, 100]  \n2018-02-27         True        False        False          False  \n2018-02-26        False         True        False          False  \n2018-02-23         True        False        False          False  \n2018-02-22         True        False        False          False  \n2018-02-14         True        False        False          False  \n...                 ...          ...          ...            ...  \n2015-03-06        False        False        False           True  \n2015-03-05         True        False        False          False  \n2015-03-04         True        False        False          False  \n2015-03-03         True        False        False          False  \n2015-03-02         True        False        False          False  \n\n[643 rows x 8 columns]",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>rise_(-100, -7]</th>\n      <th>rise_(-7, -5]</th>\n      <th>rise_(-5, -3]</th>\n      <th>rise_(-3, 0]</th>\n      <th>rise_(0, 3]</th>\n      <th>rise_(3, 5]</th>\n      <th>rise_(5, 7]</th>\n      <th>rise_(7, 100]</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>2018-02-27</th>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>True</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>2018-02-26</th>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>True</td>\n      <td>False</td>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>2018-02-23</th>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>True</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>2018-02-22</th>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>True</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>2018-02-14</th>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>True</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>2015-03-06</th>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>True</td>\n    </tr>\n    <tr>\n      <th>2015-03-05</th>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>True</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>2015-03-04</th>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>True</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>2015-03-03</th>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>True</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>2015-03-02</th>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>True</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n    </tr>\n  </tbody>\n</table>\n<p>643 rows × 8 columns</p>\n</div>"
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 得出one-hot编码矩阵\n",
    "dummies = pd.get_dummies(p_counts, prefix=\"rise\")\n",
    "dummies"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-02-22T09:47:44.213810500Z",
     "start_time": "2024-02-22T09:47:44.207859100Z"
    }
   },
   "id": "60088b4b33c05f11",
   "execution_count": 16
  },
  {
   "cell_type": "markdown",
   "source": [
    "4 小结\n",
    "- 数据离散化【知道】\n",
    "    - 可以用来减少给定连续属性值的个数\n",
    "    - 在连续属性的值域上，将值域划分为若干个离散的区间，最后用不同的符号或整数值代表落在每个子区间中的属性值。\n",
    "- qcut、cut实现数据分组【知道】\n",
    "    - qcut:大致分为相同的几组\n",
    "    - cut:自定义分组区间\n",
    "- get_dummies实现哑变量矩阵【知道】"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "c46b5ccb6d956ed4"
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
